The current mainstream sequence labeling is based on Recurrent Neural Network (RNN). Aiming at the problem of RNN and sequence labeling, an improved multilayer Bi-direction Long Short Term Memory (BLSTM) network for sequence labeling was proposed. Each layer of BLSTM had an operation of information fusion, and the output contained more contextual information. In addition, a method to perform Chinese word segmentation and punctuation prediction jointly was proposed. Experiments on the public datasets show that the improved multilayer BLSTM network model can improve the classification accuracy of Chinese segmentation and punctuation prediction. In the case of two tasks that need to be accomplished, the joint task method can greatly reduce the complexity of the system, and the new model and the joint task method can also be applied to solve other sequence labeling problems.